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ABSTRACT 

In 1993 the National Science Foundation awarded a 
Louisiana Parish a grant to fund a program to enhance science 
education in the elementary schools in the school district. Two 
teachers in each elementary school were to be trained as science 
mentors or lead teachers receiving instruction in physical science 
content, hands-on science programs, and training in alternative 
assessment. These teachers should dissem/.nate this information in 
their schools. An evaluation of the summer training program for the 
first 34 science mentors used pretest and posttest scores on a 
measure of science knowledge, a consumer satisfaction form, 
observation during the training, and interviews with program staff. 
Average scores for teacher perceptions of their training on the 
content dimensions of science were very positive, and posttest scores 
indicated increased knowledge. Teacher attitudes about the program 
were generally positive, but some recommendations were made for 
program improvement based on teacher responses. These include greater 
emphasis on alternative assessment training, and additional training 
on some teaching methods and strategies. Seven appendixes list goals, 
contain the questionnaire, and provide information about the code 
structure and frequencies for strengths and weaknesses, along with 
posttest results. (Contains 7 references.) (SLD) 
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I. Introduction 

A. Background of the Louisiana Physical Science Program 

In the Summer of 1993, the National Science Foundation awarded a Louisiana Parish a 
five year $1.7 million program entitled Louisiana Physical Science (LPS). The overall goal 
of the program is to enhance science education in the elementary schools (focusing on 
kindergarten through third grade classrooms) in the district. 

During the five years of the project, two teachers in each elementary school in the 
district will be trained either as a Science Mentor or a Lead Teacher. In these roles, the 
teachers will receive instruction in physical science content, traini'ng on how to manage and 
direct a hands-on, activity-based science program and training in using alternative methods of 
assessment in science classrocms (LPSS, 1992). To facilitate this teacher training, a series of 
summer programs have been organized to address each of these areas. It is the methodology 
and results of the Summer 1993 Program Evaluation that will be the primary focus of this 
paper. 

The LPS grant calls for the gradual diffusion of the proposed innovations in science 
teaching throughout the entire district. The grant first funded the training of a corps of 34 
Science Mentors during the 1993-94 fiscal year. These Science Mentors will then work with 
the science teachers in all of the public elementary schools in the district. After the first two 
years, the program will concentrate on the training of 62 Lead Teachers, who will in turn 
provide training to other teachers in their schools (LPSS, 1992). 

An examination of the LPS grant proposal identified seven primary objectives and 31 
activities designed to accomplish these objectives (K. T. Associates, 1993). The objectives of 

1 



the project, broken out into five teacher-oriented and two smdent-oriented objectives, arc as 
follows: 

Teacher Goal 1 . To increase the teacher's knowledge of physical science content. 
Teacher Goal 2 . To train the teachers to manage and direct a hands-on, activity- based 
science program. 

Teacher Goal 3 . To train teachers to use alternative methods of assessment. 
Teacher Goal 4 . To develop a network of Science Mentors and Lead Teachers that 
provides collegial support and leadership to each other and to other teachers in their schools. 

Teacher Goal 5 . To develop a shared vision for the science program among parents 
and the community that supports the continuance of that program. 

Student Goal 1 . To develop students' understanding of the habits of mind identified in 
Science for All Americans (Rutherford and Ahlgren, 1990). 

Student Goal 2 . To develop students' understanding of the basic concepts of the nine 
topics outlined in the LPS program. 

Activities of the Summer 1993 Program and the Academic year 1993-94 Program 
concentrated on the attainment of Teacher Goals 1, 2, and 3, plus a partial start on Teacher 
Goal 5. Neither of the Student Goals were assessed during the first year of the program, 
during which the emphasis was on staff development. A complete list of the Teacher Goals 
and Activities is found in Appendix 1, with highlights by those relevant for Academic Year 
1993-94 Program. 
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B. Description of the Summer 1993 Program 

The 34 Science Mentors met on a college campus for a three week Summer Program 
beginning July 12, 1993. Physical science content covered in the Summer 1993 Program 
centered on four topics: Magnets and Magnetism, Sound, Color and Light, and Matter and Its 
Changes. 

During the three week course, the Science Mentors were taught physical science 
content using the same strategies and techniques that they were expected to use in their 
classrooms during the 1993-94 school year. The inquiry-based lessons were designed to 
feature hands-on activities, open-ended discussions and opportunities for independent 
investigation (LPSS, 1992). 

The physical science content course was taught by a trainer from the American 
Institute of Physics and utilized materials from the Evlyn J. Daniel Educational Foundation. 
The Summer 1993 Program included having the Science Mentors work with a student . 
population of 32 children from 16 participating schools in classroom settings on the college 
campus. 

Seminar topics for the Summer 1993 Program included assessment strategies, 
classroom management, computers in the classroom, an integrated mathematics curriculum 
and relevant special education topics. The sessions were taught by recognized experts on the 
discussion topics and by LPS staff members. The Science Mentors were broken out into two 
groups to attend the seminars: kindergarten and first grade teachers in one group, second and 
third grade teachers in another group. 
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C. Goals of the Summer 1993 Program 

With regard to the Summer 1993 Program, there were three relevant goals, and three 
activities, to evaluate. These were as follows: 

(1) Teacher Goal 1 . To increase the teacher's knowledge of physical science content. 

Teacher Activity la. Increase teachers' knowledge of physical science content 
by conducting two Summer programs (Year One, Year Two) for the Science Mentors. 

(2) Teacher Goal 2 To train the teachers to manage and direct a hands-on, activity- 
based science program. 

Teacher Activity 2a. Train the teachers to manage and direct a hands-on, 
activity-based science program by conducting two Summer programs (Year One, Year 
Two) for the Science Mentors. 

(3) Teacher Goal 3 To train the teachers to use alternative methods of assessment. 

Teacher Activity 3a. Train the teachers to use alternative methods of 
assessment by conducting two Summer programs (Year One, Year Two) for the 
Science Mentors. 



IL Evaluation Design and Methodology 

A. Evaluation Design 

The evaluation design called for the use of pre- and posttest scores on a knowledge 
test developed to assess the four content areas taught during the Summer 1993 Program under 
Teacher Activity la. A consumer satisfaction form, known as the Science Mentor 
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Questionnaire (Appendix 2), was developed to assess the accomplishment of Teacher 
Activities 2a and 3a. This consumer satisfaction form was used to ascertain how adequate the 
Summer 1993 Program was in training the teachers to develop their own hands-on, activity- 
based lessons and in training them to use alternative methods of assessment. This evaluation 
design also utilized observations during the training sessions and interviews with the LPS 
Summer Program staff as sources of data. 
B. Methodology 

1. The Science Mentor Questionnaire 

The Science Mentor Questionnaire (SMQ) was developed by the LPS staff and 
evaluation team for use during the Summer 1993 Program and contains a combination of 
closed-and open-ended questions. The closed-ended items include the following: 

(1) Four items assessing the effectiveness of the training that the Science Mentors 
received in the four physical science content areas taught at the Summer 1993 Program: 
matter, sound, magnets and light/color. The responses to these items were five point scales 
with 5 indicating "highly effective". (See the SMQ, Part I in Appendix 2.) 

(2) Six items assessing the Science Mentors' perceived ability to demonstrate skills in 
teaching methods and strategies based on their participation in the Summer 1993 Program. 
The six teaching methods and strategies that were taught at the Summer 1993 Program were: 
cooperative learning, reflective logs, use of computers in the science classroom, 
accommodating individual differences, higher order thinking skills and authentic and/or 
performance-based assessment. The responses to these items were five point scales with 5 
indicating "well prepared". (See the SMQ, Part II in Appendix 2.) 



(3) Ten items assessing the Science Mentors' perception of the emphasis placed on 
different types of learning. The ten types of learning that were taught at the Summer 1993 
Program were: learning factual information; developing concepts; understanding and applying 
principles and rules; understanding and applying theory; critical analysis and/or problem 
solving; creative thinking; developing knowledge of self and others; developing professional, 
career and job-related skills; developing written communication skills; and developing oral 
communication skills. 

The responses to these items were three point scales with 3 indicating "a lot of 
emphasis". (See the SMQ, Part III in Appendix 2.) 

(4) Two items assessing the Science Mentors' opinions regarding two issues related to 
the overall training program: did the teaching methods stimulate their interest in the course; 
and were thought-provoking questions asked. The responses to these items were five point 
scales with 5 indicating "strongly agree". (See the SMQ, Part IV in Appendix 2.) 

(5) One item assessing the Science Mentors' opinion regarding the overall quality of 
teaching in the course (Summer 1993 Program). The responses to this item was a five point 
scale with 5 indicating "excellent". (See the SMQ, Part V in Appendix 2.) 

The Science Mentors were also asked open-ended questions about their perceptions of 
the strengths and weaknesses of the Summer 1993 Program, their suggestions for ways that 
the program could be improved and their perceptions regarding the presentations of the 
resource persons. These items are located in Part VI of the SMQ in Appendix 2. 

Since the data from the surveys were both numeric (answers to the closed-ended 
items) and narrative (answers to the open-ended questions) in nature, a combination of 
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quantitative and qualitative analyses were planned. The quantitative analyses included 
descriptive statistical analyses of the numeric data using procedures from the SAS data 
analysis system. Narrative data were analyzed using the constant comparative technique 
(Lincoln & Guba,1985), which was automated using the Qualpro text management system 
(Blackman, 1993). 

On the closed-ended items, descriptive statistics were first run, including frequencies, 
means and standard deviations (sds) for the survey items. There were 34 respondents to the 
SMQ, which represents the entire population of the Science Mentors who attended the 
Summer 1993 Program. The means and standard deviations for each of these 23 items are 
included in Appendix 3. The narrative data generated from the open-ended items were 
analyzed using the constant comparative method, which was automated using Qualpro 
(Blackman, 1993; Lincoln & Guba, 1985). The code structures and frequencies for the three 
open-ended items aie found in Appendices 4-6. 

2, Interview and Observation Data 

Interviews with LPS staff members and observations on-site by the evaluation team 
resulted in several tentative conclusions: 

* In general, the LPS staff appeared to have done a very good job of selecting 
somewhat knowledgeable and enthusiastic Science Mentors for the program. 

* The Science Mentors were very impressed with the abilities of the primary trainer 
for the Summer 1993 Program and wished that they could have spent more time with her. 

* The Science Mentors had a much more mixed reaction to the Seminar presentations 
with several of them complaining that some of the presentations were irrelevant or redundant 
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* The Science Mentor came to the program with a variety of backgrounds in physical 
science content and in basic knowledge regarding hands-on activities in the classroom and 
alternative forms of assessment. This resulted in some teachers being bored at times and 
others feeling overwhelmed. 

III. Results 

A. Science Mentor Questionnaire 

The means for the closed-ended items completed by the Science Mentors were 
generally positive. The average scores for their perception of their training on the four 
content areas were very positive. 

The scores were somewhat less positive for the items measuring the Science Mentors' 
perception of their preparation in six areas concerned with teaching methods and strategies. 
The Science Mentors gave high ratings to their Preparation on Cooperative Learning and on 
Higher Order Thinking Skills; moderately high ratings on their Preparation on Reflective 
Logs, on Accommodating Individual Differences and on Authentic Assessment; and a below 
average score on their Preparation on the Use of the Computer in Science Classrooms. The 
Science Mentors gave a mean score slightly above the mid-point of the scale to the i 
regarding their Preparation on Authentic Assessment (See Appendix 3). 

The open-ended items on the Science Mentor Questionnaire allowed the Science 
Mentors to make unstructured comments in three areas: their perceptions of the strengths of 
the Summer 1993 Program; their perceptions of the weaknesses of the Summer 1993 
Program; and their suggested methods of improving the program. The perceived strengths of 
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the program may be divided into three broad categories: (1) characteristics of the Summer 
1993 Program trainers and staff; (2) quality of the information taught; and (3) aspects of the 
program associated with "working with others in actual settings" (See Appendix 4). 

With regard to characteristics of the trainers and itaff, the primary trainer was the 
most frequently mentioned "strength of the program". She received 20% of the total number 
of all responses regarding strengths of the program by herself. 

With regard to the quality of the information taught, the sub-category associated with 
Content Knowledge was the most often cited, receiving 17% of all responses regarding the 
strengths of the program. This finding confirms the results from the closed-ended items noted 
above in which training in specific content ateas was rated highly by the Science Mentors. 

On the cither hand, only 5% of the responses ol the Science Mentors named 
Assessment Training as a strength of the Slimmer 1993 Program. This is problematic, since 
one of the major goals of the Summer 199 3 training was to begin training the teachers to use 
alternative methods of assessment (LPSS, I 1 ) 1 )?.; K. T Associates, 1993). 

The most common response from the Science Mentors regarding the weaknesses of the 
program was the Quality of the Resource Persons, which paradoxically had also been 
mentioned as a strength of the program. Slightly ovei 34% of the responses regarding 
program weaknesses concerned the Quality of the Resource Persons. On another open-ended 
item regarding the resource persons, the most frequently criticized areas were the computer 
training, the integrated math training and the special education topics (See Appendix 5). 

Several of the cited weaknesses concerned the need for a greater emphasis on some 
aspect of the program. The two most common responses in this category were the Need for 
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More Content Knowledge and the Need for More Information on Teaching Methods and 
Strategies. Thus, while the Science Mentors saw the teaching of physical science content as 
one of the major strengths of the program, they wanted the program to emphasize it even 
more. Part of the reason for this was the Science Mentors' stated insecurity about their level 
of knowledge in the physical sciences, which several said was not stressed in their college 
preparation programs. 

Several of the responses regarding the weaknesses of the program concerned the 
Sessions being Too Long and there being Too Much Material, with these two sub-categories 
accounting for 17% of the responses regarding weaknesses of the program. Another 
perceived weakness of the Summer 1993 Program concerned the stress that some Science 
Mentors felt in preparing lessons and authentic assessments during the training course. 

There were three broad categories of suggested methods for improving the program: 
(1) Changes in the Topics Covered; (2) Changes in the Teaching Methods Employed; and (3) 
Changes in Trainers and Resource Persons. Under Changes in Topics covered, two sub- 
categories (More Content Knowledge, More Demonstration of Methods) were the most 
frequently given responses, accounting for 34% of the total responses concerning suggested 
changes. 

Another sub-category under Changes in Topics Covered was labelled More Authentic 
Assessment, and 6% of the total number of were made on this topic. This finding is 
consistent with other data indicating that several of the Science Mentors were not satisfied 
with their training in authentic and/or performance based assessment and wanted future 
Summer Programs to cover this topic much more extensively. 
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The two most frequently given responses concerning Changes in the Teaching 
Methods Employed were More "Hands-On" Experiences and More "Sharing" Time. 
Together, these two sub-categories accounted for 23% of the total number of responses 
regarding suggestions for improving the program. Under the broad category entitled Changes 
in Trainers and Resource Persons, the most commonly gi v en suggestion was to have More 
Teacher Involvement, meaning more involvement of the Science Mentors themselves on the 
Summer Program (See Appendix 6). 
B. Pretest and Posttest Content Area 

In order to determine if the Science Mentors' knowledge of physical science content 
was enhanced as a result of attending the Summer 1993 Program, the primary trainer 
administered pretests and posttests to all participants. These pretests and posttests were open- 
ended items. 

The 34 Science Mentors were given 30 grades on the pretest and the posttest. These 
30 content areas may be found in Appendix 7. These content areas relate to Magnets and 
Magnetism (12 content areas), Color and Light (14 content areas) and Sound (4 content 
areas). None of the content areas addressed Matter and Its Changes, which was one of the 
designated topics to be covered in the Summer 1993 Program. 

As indicated in Appendix 7, Science Mentors did considerably better on the posttest 
than on the pretest. The results may be summarized as follows: 

(1) There were positive difference scores for aU the content areas, indicating that the 
average posttest for the Science Mentor was higher than the average pretest scores on all 
items. 
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(2) The difference was statistically significant (using the matched-pair t-test) for 26 of 
the 30 content areas. Of course, statistics books (e.g., Hinkle, Wiersma and Jurs, 1988) 
caution against using multiple t-tests due to the chance of reporting spuriously significant 
results. However, in this case the large number of significant results (87% of the total 
number possible) could not be due simply to violating the overall alpha level for protection 
against Type I errors. 

(3) The differences are highly significant (g < .0001) for 21 of the 30 comparisons. 
These results indicate that the training from the Summer 1993 Program had a positive 

effect on the Science Mentors' knowledge of physical science concepts. 

IV. Recommendations for Improving the Summer 1994 Program 
Based on conclusions from the various data sources described above, the following 
eight recommendations were made with regard to improving the Summer 1994 Program: 

1) Place greater emphasis on training in methods of alternative assessment (authentic 
and/or performance-based) in the Summer 1994 Program. 

2) Provide greater opportunity for the Science Mentors to practice the alternative 
methods of assessment during the Summer 1994 Program. 

3) Provide continued training in content areas, including sending reading material to 
Science Mentors (in the five areas to be emphasized in the Summer 1994 Program) in 
advance of the program. 
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4) Provide the Science Mentors with more intensive training on those teaching 
methods and strategies that were identified as insufficiently covered in the Summer 1993 
Program. 

5) Select and train seminar presenters who will deliver sessions that are relevant to 
the needs of the majority of the Science Mentors in an interesting and non-redundant manner. 

6) Allow the Science Mentors themselves more involvement in the training, 
particularly in the M hands-on M activities. 

7) Make the evaluation team responsible for the assessment of knowledge gains 
during the Summer 1994 Program. 

8) Make changes in the proposed Summer 1994 Program schedule that would satisfy 
legitimate criticisms received from the Science Mentors' assessment of the 1993 program. 
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Appendix L 

List of Louisiana Physical Science Teacher Goals and Activities: Complete for the Five 
Years of the Project 

This list was developed from a reading of the LPS 1992 proposal to the National Science 
Foundation and was included in the 1993 evaluation proposal developed by K. T. Associates. 
For Academic Year 1993-94, there were four relevant activities: Teacher Activities lb, 2b, 3b, 
and Sa. 



Teacher Goal 1 . To increase the teacher's knowledge of 
physical science content . 

Teacher Activity la . Increase teachers' knowledge of 
physical science content by conducting two Summer programs 
(Year One, Year Two) for the Science Mentor Teachers, and 
three Summer programs (Year Three, Year Four, Year Five) for 
the Science Lead Teachers. 

**** Teacher Activity lb . Conduct periodic small group 
meetings between the Science Specialists and Mentor Teachers 
to assist in development of teachers' knowledge (Year One, 
Year Two) . 

Teacher Activity lc . Conduct periodic small group 
meetings between the Science Mentor Teachers and Lead 
Teachers to assist in development of teachers' knowledge 
(Year Three, Year Four, Year Five) . 

Teacher Goal 2 . To train the teachers to manage and direct 
a hands-on, activity-based science program. 

Teacher Activity 2a . Train the teachers to manage and 
direct a hands-on, activity-based science program by 
conducting two Summer programs (Year One, Year Two) for the 
Science Mentor Teachers, and three Summer programs (Year 
Three, Year Four, Year Five) for the Science Lead Teachers. 
**** Teacher Activity 2b . Conduct periodic small group 
meetings between the Science Specialists and Mentor Teachers 
to assist in the training of teachers to manage and direct a 
hands-on, activity-based science program (Year One, Year 
Two) . 

Teacher Activity 2c . Conduct periodic small group 
meetings between the Science Mentor Teachers and Lead 
Teachers to assist in the training of teachers to manage and 
direct a hands-on, activity-based science program (Year 
Three, Year Four, Year Five) . 

Teacher Goal 3 . To train the teachers to use alternative 
methods of assessment. 

Teacher Activity 3a . Train the teachers to use 
alternative methods of assessment by conducting two Summer 
programs (Year One, Year Two) for the Science Mentor 




Teachers, and three Summer Programs (Year Three, Year Four, 
Year Five) for the Science Lead Teachers. 
**** Teacher Activity 3b . Conduct periodic small group 
meetings between the Assessment Specialist and Science 
Mentor teachers to train the teachers to use alternative 
methods of assessment (Year One, Year Two) . 

Teacher Activity 3c . Conduct periodic small group 
meetings between the Science Mentor Teachers and Lead- 
Teachers to train the teachers to use alternative methods of 
assessment (Year Three, Year Four, Year Five) . 

Teacher Goal 4 . To develop a network of Science Mentor 
Teachers and Lead Teachers that provides collegial support 
and leadership to each other and to other teachers in their 
schools . 

Teacher Activity 4a . Conduct periodic small group 
meetings between the Science Mentor Teachers and Lead 
Teachers designed to foster collegial support and mutual 
training (Year Three, Year Four, Year Five) . 

Teacher Activity 4b . Conduct a three-day session at 
the end of Year Four, focusing on the development of 
interdependent relationships among Science Mentor Teachers 
and Lead Teachers . 

Teacher Activity 4c . Conduct a three-day session at 
the end of Year Five, focusing on maintaining lines of 
communication between Science Mentor Teachers and Lead 
Teachers . 

Teacher Goal 5 . To develop a shared vision for the science 
program among parents and the community that supports the 
continuance of that program. 

**** Teacher Activity 5a . Administer the Primarily Science 
Learning Community Survey to parents and school advisory 
councils to identify strengths/weaknesses in the development 
of a shared vision at the end of Year One. 

Teacher Activity 5b . Train Science Mentor Teachers in 
Family Science (an innovative national outreach program that 
seeks to increase the study of science by elementary 
students) during the Summer of Year Two. 

Teacher Activity 5c . Provide Science Mentor Teachers 
with training that will enable them to become Family Science 
Leader Trainers during the Summer of Year Three. 

Teacher Activity 5d . Family Science Leader Trainers 
(who are also Science Mentor Teachers) will provide Science 
Lead Teachers with Family Science training during Years Four 
and Five . 

Teacher Activity Goal 5e . Family Science workshops 
will be conducted for parents and their children during 
Years Two through Five. 
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BASC 6001 COURSE EVALUATION NAME 

JULY 30, 1993 

SS# 

Part I 

How effective was the training you received in each of the four 
content areas? Please circle the appropriate response. 

MATTER 

5 = HIGHLY EFFECTIVE 
4 

3 = AVERAGE 
2 

1 = HIGHLY INEFFECTIVE 
SOUND 

5 = HIGHLY EFFECTIVE 
4 

3 = AVERAGE 
2 

1 = HIGHLY INEFFECTIVE 
MAGNETS 

5 = HIGHLY EFFECTIVE 
4 

3 = AVERAGE 
2 

1 = HIGHLY INEFFECTIVE 
LIGHT AND COLOR 

5 = HIGHLY EFFECTIVE 
4 

3 = AVERAGE 
2 

1 = HIGHLY INEFFECTIVE 
Part II 

How well prepared are you to demonstrate the following skills 
based on participation in this course? 
COOPERATIVE LEARNING 

5 = WELL PREPARED 

4 = 

3 = SOMEWHAT PREPARED 

2 = 

1 = NOT PREPARED AT ALL 
REFLECTIVE LOG 

5 = WELL PREPARED 

4 = 

3 = SOMEWHAT PREPARED 

2 = 

1 = NOT PREPARED AT ALL 
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USE OF COMPUTER IN THE SCIENCE CLASSROOM 
5 = WELL PREPARED 

4 = 

3 = SOME r ,\/HAT PREPARED 

2 = 

1 = NOT PREPARED AT ALL 
ACCOMMODATING INDIVIDUAL DIFFERENCES 

5 = WELL PREPARED 

3 = SOMEWHAT PREPARED 

1 = NOT PREPARED AT ALL 
HIGHER ORDER THINKING SKILLS 
5 = WELL PREPARED 

3 = SOMEWHAT PREPARED 

1 = NOT PREPARED AT ALL 
AUTHENTIC AND/OR PERFORMANCE BASED ASSESSMENT 
5 = WELL PREPARED 
3 = SOMEWHAT PREPARED 

1 = NOT PREPARED AT ALL 
Part III 

How much emphasis was placed on the following types of learning? 
3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphasis at all 
Learning factual information 

3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphasis at all 
Developing concepts 

3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphasis at all 
Understanding and applying principles and rules 

3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphasis at all 
Understanding and applying theories 

3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphasis at all 
Critical analysis and/or problem solving 

3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphasis at all 
Creative thinking 

3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphanJ:; rj t all 



Developing knowledge of self and others 
3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphasis at all 

Developing professional, career, and job-related skills 

3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphasis at all 
Developing written communication skills 

3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphasis at all 
Developing oral communication skills 

3 = A lot of emphasis 

2 = Some emphasis 

1 = No much emphasis at all 
Part IV 

Teaching methods stimulate interest in the course. 
5 = Strongly agree 

4 = 

3 = Agree somewhat 

2 = 

1 = Strongly disagree 

Thought -provoking questions are asked. 

5 = Strongly agree 

4 = 

3 = Agree somewhat 

2 = 

1 - Strongly disagree 

How would you grade the quality of teaching in this course? 

5 = Excellent 

4 = Good 

3 = Average 

2 - Below average 
1 = Poor 

Part V. Open-ended Questions 

1. What are the strengths of this course? 



What are the weaknesses of this course? 



3. How could this course be improved? What specific changes 
would you make? 



9 - 
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Appendix 3 

Comparison of Mean Scores for Common Items on the Louisiana 
Physical Science Mentor Questionnaire, Summer 1993 Program and 
the Spring 1994 Mentor Questionnaire (S94MQ) 



Science Mentor 
^Uc?b LionnaiL c xueiu 


Mean Score 

oUiiuueir 

Program 


Mean Score 
Program 


Differ- 
ence 


V -L / ii7axn.xixy Oil nauLer 


ft • D U 


A R A 


fi A 


(2) Training on Sound 


4.35 


4.40 


.05 


(3) Training on Magnets 


4.76 


4.81 


.05 


(4) Training on Light and 
Color 


4 .88 


4.65 


- .23 


(5) Preparation on 
Cooperative Learning 


4.21 


4.38 


. 17 


(6) Preparation on 
Reflective Log 


3 .47 


3 .15 


- .32 


(7) Preparation on Use of 
Computer 


2.71 


2 . 96 


.25 


(8) Preparation on 
Accommodating Individual 
Differences 


3 .62 


3 .73 


. 11 


(9) Preparation on Higher 
Order Thinking Skills 


4 .00 


4.35 


.35 


(10) Preparation on 
Authentic Assessment 


3 .44 


3 . 50 


.06 


Average Across All Items 


3 .99 


4.05 


.06 



Note. The responses to items 1 through 4 were on five point 
scales with 5 indicating "highly effective" . The responses to 
items 5 through 10 were on five point scales with 5 indicating 
"well prepared" . The Summer 1993 responses were from the 34 
Mentors who enrolled in the program; the Academic Year 1993-94 
responses were from the 3 2 Mentors who were active in April -May 
1994. 



Appendix 4 

Code Structure and Frequencies for the St rengths 
of the Stammer 1993 Program 



Response Category 


Responses 


Percent 


1 ( a \ Primary Trai npr (PTM 


22 


20% 


1(b) Quality of Resource 

L \^ X. ouno v Sc / 


13 


12% 


1 (r) Oualitv of ^taff (OS) 


6 


5% 


2(a) Content Knowledge (CK) 


19 


17% 


2 (b) Teaching Methods and 
Strategies (TMS) 


17 


15% 


2(c) Assessment Training (AT) 


5 


5% 


3 (a) Working with Other 
Teachers (WOT) 


13 


12% 


3(b) "Hands On" Experience (HE) 


8 


7% 


3(c) Working with Students (WS) 


7 


6% 




110 


100% 



110 Units, 9 Codes 

Four most common responses: 
Primary Trainer 
Content Knowledge 
Teaching Methods and Strategies 
Quality of Resource Persons 
Working with Other Teachers 



(22) 



(19) 
(17) 
(13)* 
(13)* 



Note . Due to rounding, the individual percents may not add 
100%. The * means tied for 4th most common response. 
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Appendix 5 



Coda Structure and Frequencies for the Weakneeeee 
of the Summer 1993 Program 



Response Category 


Responses 


Percent 


1 Quality of Resource 
Persons (QRP) 


22 


34% 


2(a) Need More Content 
Knowledge (NCK) 


6 


9% 


2 (b) Need More Teaching Methods 
and Strategies (NTMS) 


3 


5% 


2(c) Need More Primary Trainer 
(NPT) 


3 


5% 


2 (d) Need More Training in 

Various Categories (NVC) 


5 


8% 


3(a) Sessions Too Long (STL) 


6 


9% 


3 (b) Too Much Material (TMM) 


5 


8% 


4 Stress over Assignments 
(SA) 


6 


9% 


5 Other Weaknesses (OW) 


8 


13% 




64 


100% 



64 Units, 9 Codes 

Four most common responses: 

Quality of Resource Persons (22) 

Other Weaknesses ( 8) 

Need More Content Knowledge ( 6)* 

Sessions Too Long ( 6)* 

Stress over Assignments ( 6)* 



Note . Due to rounding, the individual percents may not add 
100% . The * means tied for 3rd most common response . 



Appendix 6 

Code Structure and Frequencies for the Suggested Methods for 

Improving the Program 



Response Category 


Responses 


Percent 


1(a) More Content Knowledge 
(MCK) 


14 


18% 


1(b) More Demonstration of 
Methods (MDM) 


13 


16% 


1(c) More Authentic Assessment 
(MAA) 


5 


6% 


1(d) Cover Fewer Topics (CFT) 


3 


4% 


1(e) More Leadership Training 
(MLT) 


2 


3% 


2(a) More "Hands-On" 

Experiences (MHE) 


10 


13% 


2(b) More "Sharing" Time (MST) 


8 


10% 


2(c) Make Changes in Schedule 
(MCS) 


5 


6% 


2(d) More Work with School 
Partner (WSP) 


3 


4% 


2(e) Other Organization of 
Groups (00G) 


2 


3% 


lf^} Mnrp TpRph^r (Science 

Mentor) Involvement (MTI) 


7 


9% 


3(b) Morp Primary Trainer (MPT) 


3 


4% 


3(c) Change in Resource Persons 
(CRP) 


3 


" 4% 


3(d) More Principal Involvement 
(MPI) 


1 


1% 




79 


100% 



to 



Four most common responses: 

More Content Knowledge (M) 

More Demonstration of Met hotlss (M) 

More "Hands -On H Plxpor I om 'vi\ (10) 

Mote M Sharing" Time ( H) 

Note . Duo to roundinq, I hf« iiifliviclii.il | if »i cmmiI iiiiiy mil <uU\ 
100%. 
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Appendix 7 

Results from the Pre-Posttests of Content Knowledge 



Content Area 


Difference 
in Means 


Standard 
Error 


t value 


Signi- 
ficance 
ijevei 


Law of Magnetic 

rUIco 


.61 


.13 


4.9 


. 0001 


Not Charged 


.03 


.08 


0.4 


ns 


No Monopoles 


.65 


.15 


4.4 


. 0001 


Multiple Poles 


. 18 


. 10 


1.8 


ns 


Magnetic Domains 


.76 


. 10 


7.4 


.0001 


Magnetic 
j. nciuc_ u ion 


.74 


.14 


5.4 


. 0001 


Temporary 

Mannol" l cm 
iuciy 11 tr L. X olll 


.06 


.06 


1.0 


ns 


Relative Strength 


.68 


.11 


5.8 


.0001 


Compass as a Tool 


. 94 


.10 


9.1 


. 0001 


Pole Reversal 


.97 


.09 


10.9 


.0001 


Pole Names 


.71 


. 12 


6.1 


.0001 


Magnet vs . 
Magnetic 


.29 


.15 


2.0 


ns 


Linear Property 

f\ f" T . "i rr \\ t* 


.59 


.15 


3.8 


. 0005 


Converging Lenses 


.97 


.17 


5.8 


. 0001 


Diverging Lenses 


.41 


. 13 


3 . 1 


. 005 


Refraction of 
Light 


.74 


. 15 


5.0 


. 0001 


Image Formation 


.88 


. 14 


6 . 1 


. 0001 


Imag<? Location 


.47 


. 14 


3.5 


. 005 


Focus Point of 
Lenses 


. 91 


. 15 


6.1 


.0001 


Shadow Formation 


1 . 18 


.20 


6.0 


. 0001 


Location of Light 
Source 


.79 


. 17 


4.7 


. 0001 



Absorption of 
Light 


. oz 


1 c 

. 1 b 




n on 1 
. u u u j_ 


Reflection of 
Light 


.97 


.18 


5.5 


. 0001 


Colors of Light 


1.26 


.14 


8.9 


.0001 


Colors of Pigment 


1.32 


.15 


8.8 


.0001 


White & Black 


. 50 


.15 


3.4 


.005 


Sound Production 


.74 


.13 


5.7 


.0001 


Pitch Freouencv 


. 82 


. 15 


5 . 5 


. 0001 


Loudness/ 
Amplitude 


.29 


.12 


2.4 


.05 


Length of Sound 
Sources 


.71 


.15 


4.7 


.0001 
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